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Abstract — We study the effects of noise on the dynamics 
of a system of coupled self-propelling particles in the case 
where the coupling is time-delayed, and the delays are discrete 
and randomly generated. Previous work has demonstrated that 
the stability of a class of emerging patterns depends upon 
all moments of the time delay distribution, and predicts their 
bifurcation parameter ranges. Near the bifurcations of these 
patterns, noise may induce a transition from one type of pattern 
to another. We study the onset of these noise-induced swarm 
re-organizations by numerically simulating the system over a 
range of noise intensities and for various distributions of the 
delays. Interestingly, there is a critical noise threshold above 
which the system is forced to transition from a less organized 
state to a more organized one. We explore this phenomenon by 
quantifying this critical noise threshold, and note that transition 
time between states varies as a function of both the noise 
intensity and delay distribution. 

I. Introduction 

The dynamics of interacting multi-agent or swarming 
systems in various biological and engineering fields is ac- 
tively being studied. These systems are remarkable for their 
ability to self-organize into very diverse, complex spatio- 
temporal patterns. These studies have numerous biological 
applications with widely varying spatial and temporal scales. 
Among them are bacterial colonies, schooling fish, flocking 
birds, swarming locusts, ants, and pedestrians [1], [2], [3], 
[4], [5], [6], [7], [8]. In engineering, these studies have 
investigated systems of communicating robots [9], [10], [11], 
[12] and mobile sensor networks [13]. 

A fundamental problem for the engineering of systems 
of autonomous, communicating agents is the design of 
agent-interaction protocols to achieve robotic space-time path 
planning, consensus and cooperative functions, and other 
forms of spatio-temporal organization. A fruitful approach 
has resulted from applying the tools developed in the study 
of swarms in various biological and physical contexts to aid 
in the design of algorithms for systems of communicating 
robots. This has led to the successful use of a combination 
of inter-agent and external potentials to obtain agent orga- 
nization and cooperation; however, it must be ensured that 
the results from these methods are scalable with respect to 
the number of agents. Important applications comprise the 
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following: obstacle avoidance [11], boundary tracking [14], 
[15], environmental sensing [13], [16], decentralized target 
tracking [17], environmental consensus estimation [13], [18] 
and task allocation [19]. 

An important aspect that must be accounted for in the 
design of algorithms for the spatio-temporal organization 
of communicating robotic systems is that of time delay. 
Time delay arises in latent communication between agents, 
information processing times, hardware malfunction, as well 
as actuation lag times due to inertia. Time delays in robotic 
systems are important in the areas of consensus estimation 
[18] and task allocation, where, for example, there is a 
time delay as a consequence of the time required to switch 
between different tasks [19]. Previous work has shown the 
big impact that time delays may have in the dynamics of 
a system, such as destabilization and synchronization [20], 
[21]. Moreover, time delays have been used with success for 
control purposes [22]. The initial studies considered at most 
a few discrete time delays that are constant in time. Recent 
studies have extended the aforementioned investigations to 
consider randomly selected time delays [23], [24], [25] and 
distributed time delays, i.e., when the time evolution of 
the system is affected by its history over an extended time 
interval in its past, instead of at a discrete instants [26], [27], 
[28]. 

Robust algorithms for task planning with inter-agent and 
environmental interactions need to account for the presence 
of noise at all levels in the system. Noise in the swarm's 
dynamics introduces higher complexity in the behavior and 
may produce nansitions from one coherent pattern to an- 
other, something that may be detrimental to the algorithm's 
purposes or, to the conttary, that may be exploited to escape 
unwanted states [29], [30], [31], [32]. 

Here, we investigate a swarming model where the coupling 
between agents occurs with randomly disnibuted time delays. 
We show that the attractive coupling, non-uniform, random 
time delays and external noise intensity combine to produce 
transitions between different coherent patterns. Remarkably, 
we show that under certain conditions, noise produces transi- 
tions that increase the phase space coherence of the particles. 

II. Swarm Model 

We study the spatio-temporal dynamics of a two dimen- 
sional system of N agents under the effects of two forces: 
self-propulsion and mutual attraction. We consider that the 
attraction between agents occurs in a time delayed fashion 
due to finite communication speeds and processing times. 
The dynamics of the particles is described by the following 



governing dimensionless equations: 
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for i= l,2...,N. The 2D position and velocity vectors of 
particle i at time t are denoted by r, (f) and r, (f), respectively. 
The self-propulsion of agent ; is modeled by the term 
(l — |r,| 2 )r,-. The quantity a is called the coupling constant 
and measures the strength of attraction between agents. At 
time t, agent i is attracted to the position of agent j at the 
past time t — T,y. The N(N — 1) different time delays < Tn 
are distributed according to a distribution p(r) whose mean 
is jj. T and whose standard deviation is (7 T . In contrast to 
some of our recent work, here we allow that T,- 7 ^ T,,; i.e., 
time delays are not symmetric among pairs of agents [33], 
[34]. The form of our model is based on the normal form 
for particles near a supercritical bifurcation corresponding to 
the onset of coherent motion [35]. In addition, the functional 
form of the attractive terms may be thought of as representing 
the first term in a Taylor series around a stable equilibrium 
point of a more general time-delayed potential. Various 
models of this form have been extensively used to study the 
motion of swarms [36], [30], [37], [31], [35], [32]. Lastly, 
the term !],■(?) = (tj ; ) is a two-dimensional vector of 

stochastic white noise with intensity equal to D and such 
that (riP (t))=0 and (rjf >(f)ijf (»')) - 2DS(t - f')M« 
for i,j= 1,2,. ..N and £,k=l,2. 
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Fig. 1 . A selection of the bifurcation curves of the mean field model Eq. 
(5) in the parameter space of coupling strength a and mean time delay fl r 
for different values of the time delay standard deviation <7 T = 0.2 (a) and 
0.6 (b). In region A (a) the stationary state of the mean field is stable; this 
state corresponds to the ring state (see text) of the full swarm system Eq. 

The swarm moves uniformly in region C (a), this 'translating state' 
disappears by merging with the stationary state along the curve a\l % = 1 
(red). The stationary state undergoes a first Hopf bifurcation along the 
curve above region A (a), making the mean field adopt a 'rotating state' in 
region B (a). The stationary state has an infinite series of higher order Hopf 
bifurcations; the first few members are shown in dashed curves (green, cyan 
and magenta). Comparing (a) and (b), we see how sensitive these additional 
Hopf curves are with respect to o~ T . The marker at a = 2 and jX T = 2 denotes 
the parameter values used for our numerical investigations. (Color online). 



For Eq. (0 to be accurate, 
sufficiently large so that 



we require 

Li=i Lj=\ 



that N be 

f co R(f — t)p(t)c/t and that the swarm particles remain 
relatively close together. However, since the proximity of the 
particles is not controlled directly by any parameter, one must 
rely on numerical simulations of finite swarm populations 
to establish the parameter regimes where the approximation 
holds. 

III. Results 

In the absence of noise, Eqs. (Q]) have been shown to 
possess a rich bifurcation structure, elucidated by the use 
of the mean field approximation Eq. (f2| (Figure [TJ [33]. In 
contrast to the mean field, the full system displays bistability 
of several coherent patterns. In particular, in wide portions 
of region B in Fig. QJ, Eqs. ((TJ possesses bistability of two 
different patterns: (i) a 'ring state', where the center of mass 
of the swarm is at rest and the agents rotate both clockwise 
and counterclockwise at a constant speed and radius 1 / y/a; 
(ii) a 'rotating state', where the particles form a fairly dense 
clump and move along a circular arc at constant speed, 
with all velocity vectors approximately aligned. The initial 
alignment of the particle velocities and the width of the 
time delay distribution are instrumental in determining what 
pattern is adopted after the decay of transients. Specifically, 
decreasing initial velocity alignment and increasing width 
of the delay distribution results in a higher liklihood of 
convergence to the ring state. 
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A mean field approximation of the swarm dynamics may 
be obtained by using coordinates relative to the center 
of mass r, = R + 5r,-, for i = 1,2..., N, where R(f) = 
h Yli=\ r i(0- Following [33], we use the following distributed 
delay equation to describe the mean field of the swarm: 

R = (l - |R| 2 )R-a ( R(f)- / R(t-z)p{z)dz) . (2) 
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Fig. 2. Snapshots of the swarm agents at different times, illustrating the 
transition from the ring state to the rotating state. Here o" T =0.2 and D =0.23. 
The circle in the different panels is simply shown as a guide to the eye. 

When noise is introduced, the combination of the cou- 
pling strength, the time delay and noise produces interesting 
pattern transitions due to the fluctuations of the agent's 



alignment. Specifically, we find that when there is bistability 
of the ring and rotating state, there is a critical value of 
the noise intensity, D crlt , above which the swarm transitions 
from the ring state to the rotating state. In contrast, noise 
intensities below that critical value do not produce such a 
transition; that is, no such transition has been observed within 
the limits of our long-time numerical simulations. Figure [2] 
shows snapshots of the swarm, illustrating this transition. 
Here, and the rest of the simulations discussed below, the 
initial state of the particles is considered to be at rest and 
the particles are randomly distributed on the unit square. 
The new state of the swarm at each time step is found by 
updating the stochastic system ([1]) using Heun's Method. 
In all simulations we assume the delays T are uniformly 
distributed with a mean delay of and a standard deviation 
of c T . The noise is assumed to be Gaussian with intensity D. 
For all of our numerical studies, we use the values N = 50, 
a = 2 and ji T = 2. 

A remarkable fact is that noise intensities D > D cr u 
produce a transition from a less coherent state into another 
with higher coherence. This is because the ring state is a 
disorganized state with both position and velocity vectors 
adopting wide probability distributions. In contrast, the ro- 
tating state is highly coherent, with particles having nearby 
locations (high density swarms) and almost perfect velocity 
alignment. 
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Fig. 3. Time series showing the mean particle alignment as a function of 
time, for different parameter values. The top row has time delay standard 
deviation o~ T =0.2 and noise intensities D =0.25 (a) and 0.4 (b). The bottom 
row has time delay standard deviation ff T =0.6 and noise intensities D =0.25 
(c) and 0.4 (d). Please note the different scales on the abcissa axis. 



In order to properly quantify the time required by the 
swarm to make this transition, we use a quantity called the 
mean alignment, which has been used in the past to describe 
the pattern adopted by the swarm as a whole [32]. This 
quantity is defined as follows. If the velocity of particle j 
makes an angle 6j with the velocity of the center of mass, 
then the mean alignment is simply the ensemble average of 



the cosines of all of the angles 9j, for j = 1,2... N. That is, 

1 " l n r r 

Mean swarm alignment = — > cosf?, = — ) — — — , 

N h ^tiNI R l 

(3) 

which ranges from -1 to 1. When all particles have per- 
fectly aligned velocities, the mean alignment is equal to 1, 
regardless of their location in space or the magnitude of the 
individual particles' speeds. 

Figure [3] shows the mean particle alignment as a function 
of time, for different values of the noise intensity and the 
time delay standard deviation. The four panels show how the 
swarm starts with very low values of the mean alignment, 
since it initially adopts the ring state. After some dwell time 
in the ring state, the noise causes the swarm depart from 
that state and converge to the rotating state, where the mean 
alignment is almost 1.0. Once the agents begin to depart from 
the ring state, the transition time required to complete the 
transition is very short compared to the dwell time. For the 
simulations shown, the dwell time decreases with increasing 
noise intensity (Fig. [3^ to [3b and Fig. [3]: to[3ji) and increases 
with increasing time delay standard deviation (Fig. [3^ to [5J: 
and Fig. [3b to Ell). 

To better understand the onset of alignment due to noise, 
we probe different noise intensities for various choices of 
<7 T , with fixed, and then plot their asymptoptic mean 
alignment Figure |4] Each of the numerical experiments in 
Figure |4] was started in an initial state that, outside of the 
presence of noise, will stay indefinitely in the ring formation 
(unaligned). The simulation was run out to t = 2500, in order 
to allow transients to pass, and to ensure we capture any 
transition between the two states. As reported in [32], we 
do see a critical value of the noise intensity D at which 
the noise drives the particles into the highly aligned rotating 
state. Interestingly, as a T is increased, the location of the D cn y 
shifts so that a larger noise intensity is required to observe 
this transition. Thus, we observe a dependence of the critical 
noise threshold on the distribution width of the delays. 
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Fig. 4. Asymptotic mean particle alignment as a function of the noise 
intensity parameter, D, and for different values of the time delay standard 
deviation o" T . Each curve (labeling different values of the standard deviation 
of the time delay ) has an extremely sharp transition that occurs at a critical 
value of the noise intensity Dcrit W-2 < D cr j, < 0.25). Below the transition 
point the swarm converges to the ring state and remains there for times as 
long as our simulations permit. In contrast, above the critical noise value, 
the swarm transitions to the more coherent, rotating sate. 

Further, as observed in Fig. [3] increasing c T also increases 
the amount of time the particles remain in the ring state 



before finally transitioning to the rotating state (assuming 
D is large enough for such a transition to occur). To better 
understand this observation, we do single runs for various 
values of a z and D. For each simulation, we monitor when 
a threshold value of the mean alignment is reached, and then 
record that time as the 'dwell time'. While these dwell time 
values, shown in figure |5j record only single events, they do 
indicate that, generally speaking, below the threshold value 
D < D cr it{a x ), we do not observe a transition out of the ring 
state, but for D > D cnr ((7 T ), we observe that the dwell time 
is highest near the threshold, and then rapidly decreases as 
D is increased. As one continues to increase D even further, 
the rotating state will gradually become more disorganized 
until noise dominates the entire system. 
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Fig. 5. Dwelling time in the ring state, as a function of noise intensity, 
D, and the time delay standard deviation, o~ T . Below the o" T -dependent, 
critical value of noise intensity, D cr j,, the transition times diverge from 
the perspective of our long-time numerical simulations (white region). 
Above the value Z) c „ f , the transition times become finite and accessible 
to computation from numerical simulations. 

IV. Discussion 

We have considered the general problem of multi-agent 
swarms of particles where the communication is governed 
by a delay coupled potential field. In particular, we have 
considered the case where the delay coupling is fixed in time 
but randomly distributed from a chosen probability density. 
This corresponds to the fact that in many cases the delays 
in signal transmission and/or reception are caused by finite 
transmission times, processing and control delays, and the 
probability of dropped packets. Modeling the swarm as a 
globally coupled delay system, we have considered the role 
of temporal noise due to fluctuations from external forces 
which occur when the swarm is operating in a random 
field. In particular, we have considered parameters of the 
distributed delay system in which there exists bi-stability; 
i.e., there co-exists both a ring state and a rotating state. 

For a given noise density and delay distribution, we have 
characterized the observation that a specific range of noise 
intensities forces a swarm from a disordered ring state, to 
a more ordered rotating state. By probing the effects of 
noise intensity along with the delay distribution width, we 
see a two parameter set which describes fluctuations which 
cause switching between disordered and ordered states. In 
particular, fluctuations due to sufficient noise intensities are 
observed to produce highly coherent and compact structures, 
which is clearly a non-intuitive result. We note once more 



that the patterns and the transitions between them, do not 
fundamentally change with the addition of small, local 
repulsive forces between particles. Stronger repulsion can, 
however, destabilize the coherent structures. 

Understanding these types noise-induced transitions is 
key to preventing coherence collapse in delay-coupled au- 
tonomous systems, as well as formulating control strategies. 
In particular, the idea of a critical noise threshold which 
can serve to facilitate transitions between different dynamic 
patterns is very interesting and powerful, and understanding 
its dependence on the structure of delay-coupled systems is 
an area of ongoing interest. 

We note that in the future, more work is required to 
understand the role of fluctuations on delay coupled systems. 
Towards this end, a general theory of switching for non- 
Gaussian noise is needed. In addition, more systematic 
numerical simulations will allow sufficient averaging to 
refine our understanding of the transitions discussed here. 
In addition, realistic networks of communication need to 
be considered beyond the globally coupled system modeled 
here. 
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